Calculation of a distribution free estimate of effect size and confidence intervals using VBA/Excel

Reporting effect sizes aids the transparent presentation and independent interpretation of scientific data. However, calculation and reporting of effect sizes for data obtained in basic research is rare. A standardized effect size was reported by Norman Cliff, known as Cliff's delta. It has several advantageous features, as (i) it makes no assumption on the shape of the underlying distribution, (ii) it works well for small to moderate samples (n>10), (iii) it is easy to calculate, and (iv) its basis is readily understood by non statisticians. Here, a VBA macro, implemented in Excel, is presented. The macro takes two independent samples as input and calculates Cliff's delta with 95% confidence intervals. The macro will reduce the barrier for calculating the effect size and can be a valuable tool for research and teaching. Introduction The use of Null Hypothesis Significance Testing (NHST) for evaluation of scientific data has been highly debated (Goodman, 2008; Cumming, 2014; Nuzzo, 2014). Several papers have highlighted misinterpretation of NHST and resulting p-values (Goodman, 2008; Halsey et al., 2015; Ivarsson et al., 2015; Wasserstein and Lazar, 2016) and have called for use of estimation statistics as alternative (Nakagawa and Cuthill, 2007; Cumming, 2014; Claridge-Chang and Assam, 2016) or additional (Drummond and Tom, 2012; Sullivan and Feinn, 2012) strategy for data analysis and presentation. Here, I only treat the case in which the data is obtained from a randomized experiment on two independent groups. The NHST returns a p-value that indicates the probability that the data from the two groups is identical, i.e. the null hypothesis is true, given the observed data or more extreme values. If the p-value is below a predefined, arbitrary threshold, usually p<0.05, the result is explained as evidence in favor of an alternative hypothesis, . CC-BY 4.0 International license not peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was . http://dx.doi.org/10.1101/073999 doi: bioRxiv preprint first posted online Sep. 8, 2016;